Adaptive Probabilistic Trajectory Optimization via Efficient Approximate Inference

Robotic systems must be able to quickly and robustly make decisions when op-erating in uncertain and dynamic environments. While Reinforcement Learning(RL) can be used to compute optimal policies with little prior knowledge about theenvironment, it suffers from slow convergence. An alternative approach is ModelPredictive Control (MPC), which optimizes policies quickly, but also requiresaccurate models of the system dynamics and environment. In this paper we proposea new approach, adaptive probabilistic trajectory optimization, that combines thebenefits of RL and MPC. Our method uses scalable approximate inference to learnand updates probabilistic models in an online incremental fashion while also com-puting optimal control policies via successive local approximations. We presenttwo variations of our algorithm based on the Sparse Spectrum Gaussian Process(SSGP) model, and we test our algorithm on three learning tasks, demonstratingthe effectiveness and efficiency of our approach.